Asynchronous Logging and Fast Recovery for a Large-Scale Distributed In-Memory Storage
نویسندگان
چکیده
Large-scale interactive applications and online graph analytic processing require very fast data access to many small data objects. DXRAM addresses these challenges by keeping all data always in memory of potentially many nodes aggregated in a data center. Data loss in case of node failures is prevented by an asynchronous logging on flash disks. In this paper we present the architecture of a novel logging service designed to support billions of small data objects, flash disk characteristics and fast node recovery. The latter is challenging if 32-64 GB of in-memory data of a failed node needs to be recovered in seconds from the logs.
منابع مشابه
New Causal Message Logging Protocol with Asynchronous Checkpointing for Distributed Systems
Causal message logging is an efficient approach for tolerating failures of processes in distributed systems because it has the advantages of both pessimistic and optimistic message logging approach. However, traditional causal message logging protocols prevent live processes from executing continuously their computation and require some synchronous logging to the stable storage during recovery....
متن کاملLightweight Message Logging Protocol for Distributed Sensor Networks
Among a lot of rollback-recovery protocols developed for providing fault-tolerance for long-running distributed applications, sender-based message logging with checkpointing is one of the most lightweight fault-tolerance techniques to be capable of being applied in this field, significantly decreasing high failure-free overhead of synchronous logging by using message sender's volatile memory as...
متن کاملAccommodating Logical Logging under Fuzzy Checkpointing in Main Memory Databasesy
This paper presents a simple and effective method to reduce the size of log data for recovery in main memory databases. Fuzzy checkpointing is known to be very efficient in main memory databases due to asynchronous backup activities. By this feature, most recovery works in the past have used only physical logging schemes. Since the size of physical log records is quite large, physical logging s...
متن کاملAccommodating Logical Logging under Fuzzy Checkpointing in Main Memory Databases
This paper presents a simple and effective method to reduce the size of log data for recovery in main memory databases. Fuzzy checkpointing is known to be very efficient in main memory databases due to asynchronous backup activities. By this feature, most recovery works in the past have used only physical logging schemes. Since the size of physical log records is quite large, physical logging s...
متن کاملVirtual Sender-based Message Logging for Large-scale Ubiquitous Sensor Network Systems
Sender-based message logging can considerably lower high failure-free overhead of receiver-based message logging resulting from synchronously logging each message into stable storage by using volatile memory of its sender as storage for logging. This beneficial feature can be obtained at the expense of extra communication costs required for allowing message senders to get receive sequence numbe...
متن کامل